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Abstract 

We provide a compact exact representation for the distribution of 
the matrix elements of the Wishart-type random matrices A^A, for 
any finite number of rows and columns of A , without any large N ap¬ 
proximations. In particular we treat the case when the Wishart-type 
random matrix contains redundant, non-random information, which 
is a new result. This representation is of interest for a procedure of 
reconstructing the redundant information hidden in Wishart matrices, 
with potential applications to numerous models based on biological, 
social and artificial intelligence networks. 


1 Introduction 

Random matrices of the form fi = A^A, where A is a random rectangular 
matrix of size n x k occur in many applications. Introduced in the classi¬ 
cal paper by Wishart fl], (probably the first application of random matrix 
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models), they form the cornerstone of multivariate statistical analysis |2|. 
Standard applications include biology, economy, telecommunication, to men¬ 
tion a few. Usually, n samples of h-dimensional rows of data are used to 
construct the covariance matrix. Typical measurements are e.g. samples of 
large number of meteorological observations at various sites collected at time 
intervals 0, high frequency financial data for large portfolios [£ij or wireless 
channels with multiple antennae or receivers ||. Wishart matrices appear 
also in fundamental science, ranging from condensed matter physics 0, nu¬ 
clear physics 0 to chiral quantum chromodynamics || . Recently, the topic of 
retrieving the redundant information from Wishart matrices got a new twist 
due to the spectacular increase in computing and storage powers. The key 
problem is to handle effectively large incidence matrices constructed by In¬ 
telligent Retrieval (IR) engines. These new challenges vary from eliminating 
the redundant information in Internet traffic, through exploiting the hidden 
information of knowledge networks 0 to unraveling the cross-correlations in 
bioinformatics and genetics. 

Let us consider a matrix = A^A. There are two qualitatively different 
situations that may appear. When n > k, all the elements of are indeed 
random. Their probability distribution was first derived by Wishart in JlJ. In 
the opposite case n < k, there is a lot of redundancy in and only a part of 
the matrix elements are random, the remaining elements are unambiguously 
determined in terms of the random ones. The analytical expression for the 
probability distribution of matrix elements of in this ‘Anti-Wishart’ case 
(this name was coined by Yi-Cheng Zhang) remained, however, unknown. 
The numerical study of the Anti-Wishart case was presented in [D|. In this 
paper we would like to derive this probability distribution as well as to provide 
a procedure of reconstructing the redundant information from the first n rows 
of the matrix. All these results are exact for any n and k, without any large 
N approximations. 

The plan of this paper is as follows. In the next two sections we will derive 
recurrence relations for the probability distributions for complex Gaussian 
matrices A. In section 4 we will use these relations to rederive the classical 
Wishart case, and then proceed to analyze the new Anti-Wishart case. For 
completeness, in section 5 we will state some known results for the joint 
eigenvalue distributions in the Anti-Wishart case. We will end the paper with 
a discussion and two appendices which state the analogous results, when A 
are real Gaussian matrices and include some mathematical details. 
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2 Wishart and Anti-Wishart random matri¬ 
ces 

Let A be a complex rectangular matrix of size n x k taken from a Gaussian 
ensemble: 

p(A) = ^s e ~ tIAU (1) 

We would like to derive the probability distribution of the elements of the 
Wishart matrix = A^A. We thus have to evaluate 

P n , fc (ft) = [ dAdA ] S(fl - A*A) P(A) (2) 

When n > k (‘Wishart case’) the resulting distribution was obtained by 
Wishart after a quite intricate calculation for real A (see e.g. [0). The 
analogous result for complex matrices is 

P n,k SHART (P) = C n ,k (det Q) n ~ k e~ tin (3) 

where C n j- is a normalization constant. 

When n < k (‘Anti-Wishart case’) no such explicit formula was known. 
The goal of this work is to give a simple unified derivation of P n ,fc(^) which 
works in both cases and to provide a procedure of reconstructing the redun¬ 
dant information from the first n rows. The problem of reconstructing the 
matrix from more realistic sparse data will be considered in a subsequent 
work. 

As was noted in ||, the determination of P 7l) fe(0) can be easily translated, 
using the integral representation 

- A'A) = / dTe jtrT ^ u ) (4) 

into the integral 

P n ,k{V) = C' n:k J dTdet(l + iT)~ n e iirTU (5) 

where T is a k x k hermitian matrix. 
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3 Recurrence relations 


The integrals of the form (||) with fl diagonal were considered for n > k in a 
beautiful paper of Fyodorov 0- Here we will slightly generalize his proce¬ 
dure for arbitrary non-diagonal fh Although the integrals (||) are invariant 
with respect to unitary transformations, this generalization is necessary in 
the case n < k. Indeed, then the matrix H has k — n exact zero eigenvalues 
and the Jacobian for the diagonalization H = UAU 1 will be quite nontrivial. 

Let us first decompose the matrices T = T k and = fl k as 


r, = 
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( 6 ) 


We will derive a recurrence relation by first integrating over Hi and then over 
the vector t. To this end we use the identity 

det(l + iT k ) = (l + it u + f f (l + iT k ^)~ l t^ • det(l + i%~\) (7) 

The integral over tu can be done by residues giving 

J dTk-ie i ^' VTk ~ iak ~ 1 (det(l+iTk^i))~ n ujy 1 e ~ UJ11 ■ f 


The last integral is Gaussian giving 
1 


, ,k -1 

(Un 


3 t ■ det(l + iT k _ x ) ■ e~ =17“ (1+iTk ~ l)uJ 


<V wll '7r 


Substituting this back into (H) leads to 

/ dr fc _ 1 (det(l + 

Hence we are led to the recurrence relation 

P„,AVk) = P -■ to. . __L.t 


Pn—l,k —1 ( UJUJ 

u u 


( 8 ) 

(9) 

( 10 ) 

( 11 ) 


4 Probability distributions for elements of Q 

We will now use the recurrence relation (ED to determine P njk (Q). Let us 
first consider the easy ‘Wishart case’ (n > k ). Then repeated use of m 
reduces the problem to calculating P Ui i(fl) which is just 

P n ,i(to) = Lc’ n ~ 1 e~ UJ (12) 
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Since in the Wishart case the eigenvalues are generically distinct we may 
diagonalize the matrix fl, and then the recurrence relation can be solved 
immediately [fl(]] to get 


P. 


“(O) = C'" t (detfi)”-‘e- trn 


(13) 


Let us now turn to the more interesting Anti-Wishart case (n < k ). Then 
repeated use of the recurrence relation reduces to the initial condition 


Po,k{n k ) = 6{Sl k ) (14) 

is dehned on the space of hermitian matrices through the integral 

<5(n fe ) = f dAe itr nkA (15) 

where the integration domain covers the space of hermitian k x k matrices. 
In the first nontrivial case we then have 

Pi,k(n k ) = - — loluA (16) 

Due to the form of the Dirac delta function we may immediately recognize 
that the argument of the exponent is just the ordinary trace of 


Solution of the recurrence relations 


We will now obtain an explicit expression for general n < k. The recurrence 
relation (|TTD expresses the probability distribution of a k x k matrix Q*. by a 
probability distribution ofa/c — 1 xfc-1 one. The first step towards hireling 
the general solution is to obtain an explicit expression for the elements of the 
relevant k — i x k — i matrix at the i th step of the recursion. 

We therefore have to solve: 






n 


wWwW t 
(i) 

UJ\{ 


(17) 

(18) 


where the superscript (z) denotes the k — i step of the recursion while the 
subscripts are dehned through the decomposition analogous to (|6|): 


D (i) 


w 11 

iS>. 

— h 


oh) 
lL i -1 


(19) 
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The ‘reduced’ matrix fiW i s Q f size i x i. 

Once the explicit expressions for fiW are known, the probability distribu¬ 
tion for the Anti-Wishart case can be written as: 


Pn,kiS^k) 



,X k ~ i ) 

11 


h(0 (fc - n) ) 


( 20 ) 


The matrix-valued delta in (^) of the hermitian (k — n) x (k — n ) matrix 

Q (k-n) 

is equivalent to the set of [k — n ) 2 deltas for the independent matrix 
elements (keeping in mind that the diagonal elements are real and the off- 
diagonal ones appear in complex conjugate pairs). 

We will first show that the argument of the exponent is just the trace of 
0. Let us take the trace of both sides of the recurrence relation (|TB|): 


tr 0 {i_1) 


tr _ yW 


oj(*)t u/*) 

(i) 

^ii 


Then the sum in the exponent may be rewritten as 


( 21 ) 


- J2 ( tr - tr (22) 

i=0 


In this alternating sum all terms cancel except the first and the last which 
give —tr f+ tr The last matrix vanishes due to the Delta function 

in ( ]20|) , while the first one is just the original matrix D (see (|17|)) . 

In order to deal with the remaining terms in (|20|) we have to explicitly 
solve the recurrence relations ([T7| ) -(p~8| ) . 

Remarkably enough one can give a compact formula for the reduced ma¬ 
trices in terms of a ratio of determinants. Let us denote by D[j] the upper 
left hand i x i sub-matrix of the original matrix fl: 



(23) 


Furthermore for each l,m > i we will consider the ((i + 1) x {i + 1)) matrix 
obtained by adjoining the I th row and m th column of to Ou: 
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In terms of these data, the solution to (|T7j) can be expressed through the 
simple formula 

\(k—i) det 


n 


Inn 


det fh. 


(25) 


We give some details of the proof in Appendix B. From here we can easily 
read off the elements entering formula 


{k—i, 
<*>11 = 


det 


i+i] 


det Oiji 


(26) 


where deth2[ 0 ] is understood as 1. Putting together the above results, we 
obtain our final result for the probability distribution for the Anti-Wishart 
case: 


pANTI-WISHART 
^n,k 


(n) = c^(detn w ) 


n—k 


e -tro 


k k 

n n & 

l=n +1 m=n +1 


det Q[n],lm 
det fl[ n ] 


( 27 -) 

As explained above, the number of independent deltas is (k — n) 2 . 

The above formula has two important features. The Dirac delta function 
shows that part of the matrix elements of D are nonrandom and are expressed 
deterministically in terms of the first n rows of fh In fact this does not depend 
in any way on the type of randomness assumed. The reconstruction formula 
works even for a fixed D = A'A. Moreover the quotient of the determinants 
is a linear function of a;; m , thus giving a simple expression for in terms 
of the elements coming from the first n rows of In addition we note that 
given a fixed matrix = A^A empirically, with absolutely no information on 
A. we may determine the size of the A matrix by computing the successive 
reduced matrices and checking the value of i when these matrices vanish. 

The pre-factor of the Dirac delta function, which gives the probability 
distribution for the first n rows of f2 is linked with the type of randomness 
of A. 


5 Joint eigenvalue distributions 


In this section, for completeness, we give the results for the joint eigen¬ 
value distributions for Wishart and Anti-Wishart random matrices, which, 
in contrast to the probability distributions of the matrix elements themselves 
Pn,k(Qk) derived in the preceding section, are rather well-known [11]. In the 
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Wishart case (n > k) all the eigenvalues are generically distinct and the 
result follows from ([T3|) in the standard way by including a Vandermonde 
determinant 


12 


pWISHART 


(A 1 ,...,Aj=nv 


n—k—Xi 


• II(^ - X ) 2 

i<j 


(28) 


In the Anti-Wishart case there are always k — n exact zero-modes. The 
remaining n nonzero eigenvalues of A' f A are distributed with the same prob¬ 
ability distribution as the eigenvalues of AAf. This can be seen by diag¬ 
onalizing the rectangular matrix A through A = USSV with U G U(k ), 
V G U(ri) and 

SS = ( 0 ) (29) 

is a k x n matrix with A diagonal. Then AA^ = USSSS^W and A^A = 
V^SS^SSV. Hence the nonzero eigenvalues of AA* and A^A coincide. 

We may then write immediately the joint eigenvalue distribution for the 
Anti-Wishart case (n < k): 


P^ TI - WISHART (\ i,..., A n ) = n 

i 


■ n(A, - \) 2 

i<j 


(30) 


We do not discuss here the asymptotic spectral properties of (^,[3(]). For re¬ 
cent results and original references we refer to recent mathematical literature 
on this subject |L3]. 


6 Conclusions 


Using the recently advocated properties of the Ingham-Siegel [;Uj] integrals, 
we have provided a compact expression for the distribution of the matrix 
elements of the Wishart ensemble, in the case where the numbers of rows 
and columns are finite and arbitrary. We gave a unified derivation which 
encompasses both the classical Wishart case (where the number of rows is 
greater or equal to the number of columns), and the opposite case, where 
part of the matrix is necessarily nonrandom (anti-Wishart case). 

The expression obtained is a starting point for algorithms leading to fast 
reconstruction of the redundant information from the first n rows of the 
Wishart matrix. The formulas for the reduced matrices allow also to test an 







empirical Wishart-likc matrix for redundant information. The more general 
problem of reconstructing the Wishart matrix from less regularly distributed 
data points (sparse matrices) remains a challenging problem, which will be 
analyzed elsewhere. 
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Appendix A — Real Wishart and Anti-Wishart 
matrices 


In this appendix we will briefly summarize the relevant formulas for D = A^A 
where now A is taken from a Gaussian real ensemble. The determination of 
P(f2) follows the complex case, using now the integral representation 


P n . k (Q) = I dT det(1 +? r)~ n /V trTn (31) 


where we now integrate over real symmetric matrices T. Instead of the 
integration by residues leading to (|8|), we use the formula (14] 



dte itw 


1 

(a + it) v 




(32) 


Integrating over fn gives, modulo overall constants, 


J dr fc _ 1 e itrrfe - infc - 1 (det(l+ir fc _ 1 ))- n /V 1 / 2 _ 1 e - a;i1 ■ 


( 33 ) 
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The last integral is real Gaussian giving 

—s=r • [det(1 + iT fc _0]3 • e -=h- t(1 +"i- l) ” ( 34 ) 

w n 2 

Hence we are led to the recurrence relation 

P n ,k(^k) oc cn n 2 e _aJll_ V7P n _i jfc _i ( fl fe _i- uxJ ) (35) 

I 11 the classical Wishart case (real variables) using relation fl35|) repeatedly 
we may reduce the problem to calculating P nl (Q) = u> n ^ 2 ~ 1 e~ u . Again the 
recurrence relation may be solved after diagonalizing fl, and we recover the 
result 

pWisnART^ = c Zk (det n) =}=i e-trn (36) 

Similar iteration like in the complex Anti-Wishart case (19) leads to matrix¬ 
valued delta for the reduced matrix Vl^ k ~ n \ It is defined by an analogous 
formula to © but with the domain of integration being restricted to the 
space of real symmetric (k — n) x {k — n) matrices. Its (k — n)(k — n + l)/2 
independent elements can be again rewritten as ratios of determinants, thus 
arriving at the probability distribution 


P 


ANT I-WISH ART 
n,k 


m = detO [n] )^e 


-trn 


n n * 

1=71 +1 m=n+l 

Km 


'det Hi 


n],lm 


det fir. 


(37) 

The number of independent scalar deltas is equal to (k — n)(k — n + l)/2 and 
agrees with the one in |15| . 

For completeness, we remind that joint eigenvalue distributions follow 
once we include the appropriate real Vandermonde determinants: 


pWISHART 
n.k 


tdANTI-WISHART 
^n,k 


7 

(Ai,...,Afc) = 

i 

(K---An) = nv 


—A 


k — n—1 

2 g-A i 


• • ri( A i - w is*) 

i<j 

II(A.-V)' J (39) 

i<j 


where (3=1 for the real case. For recent results on spectral analysis of the 
above formula we refer to |jl3|. 
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Appendix B — Some details of the proof of (2t>). 


The proof can be done by induction on i. For i — 0 the statement is obviously 
true. Since in this case we have 


Q 


(k) 

lm 


detf][ 0 ] 




lm 


(40) 


Let us assume that the formula (^) holds for i — 1. We will verify the formula 
for i. To this end let us rewrite the recurrence relation (|I8|) 


0 (fc-i) 
1 ^lm 


q(/s— i+l) 
J L k—i\lm 


u; (fc-i+l) u; (fc-i+l)t 


U 


(k—i+1) 

11 


(41) 


as 


q(A:— i+l) q(/c— i+l) 

0 (fc-i) _ 0 (fc—i+1) _ “Z+ll “lm+1 
— I+lm+1 0 (fc-i+l) 

“l 1 


(42) 


where we made use of the decomposition 
(|25|) to obtain 


We will now insert the formula 


detOjj] ^_)_j m+i detfl[j_i] i +i m+i detfl[j_xp_|_j jdetfl[j_p im+i 


detQm 


detfl[j_i] 


det [a—i] det [&—i] a 


(43) 


When we multiply this by detf^detf^j-i] we see that the induction step is 
equivalent to proving the following determinental identity^: 


detQ[j— i ^ detfl[j]) +i rn-\-i det —i]/-f-i 77/tz d e li 1 y—^ det —ij/-{-i ^det i—l\i m+i 

(44) 

where we used the notation defined in (E3I) and (Ell). Now we use repeatedly, 


for each of the terms of (f44|) the decomposition 


det 

/ 

^[i—1] 

) 

= det Op_i] ■ (a - '0 t ^^ 1 ]'0J 

(45) 


\ + 

<y ) 




The only complication lies in applying this formula to det where terms 
involving f^ 1 do appear. To proceed we have to use a similar decomposition 

1 Note that detQ^i]^ = detfi^]. 
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for the inverse matrix: 
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%-l] 

if 

-1 
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l w 

a ) 


{ bt 

c J 


with 

a = 

b = -cn^jp 

V = -c^tt { T^!] 

1 

c — - 

(T - if^Q^lf 


After a straightforward but tedious calculation we arrive at ([44]). 


( 46 ) 


(47) 

(48) 

(49) 

(60) 
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